Cross-Modality Fusion Transformer for Multispectral Object Detection

Multispectral image pairs can provide the combined information, making object detection applications more reliable and robust in open world. To fully exploit different modalities, we present a simple yet effective cross-modality feature fusion approach, named Cross-Modality Fusion Transformer (CFT) this paper. Unlike prior CNNs-based works, guided by transformer scheme, our network learns long-range dependencies integrates global contextual information extraction stage. More importantly, leveraging self attention of transformer, naturally carry out simultaneous intra-modality inter-modality fusion, robustly capture latent interactions between RGB Thermal domains, thereby significantly improving performance multispectral detection. Extensive experiments ablation studies on multiple datasets demonstrate that approach is achieves state-of-the-art performance. Our code models are available at https://github.com/DocF/multispectral-object-detection.